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HOW EFFECTIVE IS PROGRAMMED INSTRUCTION IN TEACHING OF READING? 



by 

Edward Fry 
Reading Center 

Rutgers - The State University 
New Brunswick, New Jersey 



"Glittering prizes are offered to those who introduce 
programmed instruction into educational and training 
situations* There has been no shortage of good reasons 
why programmed instructions must he, in principle, 
superior to conventional classroom instruction, and 
indeed, the first empirical studies in the (British) 
armed forces seemed to promise drastic improvements 
in learning rate* More recent work, however, has 
tended to indicate something nearer parity between pro- 
grammed ins t miction on one hand and conventional class- 
room instruction on the other. " (4 ) 



This quotation from Duncan in 1965, speaking about the 
experiences of the British armed forces in tht* use of programmed 
instruction, could fairly well be mirrored by reading teachers who 
have seriously investigated the use of programmed instruction in the 
teaching of reading* It is not that programmed instruction cannot 
teach? there is much evidence to the fact that it can. The difficulty 
is that teachers expect it to teach better than traditional or con- 
ventional methods and this it cannot do, at least with any degree of 
consistency. But before we go into the problem of effectiveness, 
let us review some of the principles of programmed instruction which 
can also serve as a definition of what we are talking about* 
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PRINCIPLES OF PROGRAMMED INSTRUCTION 

Although there is something less than unanimous agree- 
ment on the principles /involved in a programmed instruction situa- 
tion, here are some which many would agree upon: [ 6 ) 

1. The subject matter is broken up into small units 
called frames. In actual practice, these frames 
usually vary in size from a short sentence to 
several small paragraphs. 

2. At least part of the frame requires some type of 
response from the student. He must answer a 
question or fill in a blank. Active participation 
on the part of the student is required, y Generally, 
it is desired that the activity also demonstrate 
understanding of the material. 

3. The stude*4'V# provided immediate feedback reinforce- 
ment. He is told the correctness of his answer, 
which has .the advantage of immediately reinforcing 
the activity or immediately correcting a misunder- 
standing. Since many programs are written in such 

a way that the student is right a high percentage of 
the time, the act of telling the student that he is 
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correct becomes a reward or reinforcement. Thus 
programs have a much higher amount of reward or 
reinforcement than most ordinary teaching situations. 

4. The units are arranged in careful sequence. Because 
the subject matter is broken into small bits, the 
author must think carefully about the learning steps 
involved, and the result is a much better sequence 
of presentation. Careful sequence also embodies 
the notion of shaping or gradually leading the 
student toward the desired goals by rewarding him 
for activity that more and more closely approximates 
those goals. 

5. Programs are aimed at specific goals. This has 
the desirable effect of making those involved in 
training evaluate their goals much more carefully 
and specifically. 

6. Revisions are based on student responses. Because 
the student's behavior can be recorded for each 
frame, a knowledge of his understanding of each 
part of the lesson can be easily obtained. Thus, 
if a student is making many errors on one section, 
the program obviously is not teaching well and must 
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he revised. Here, then, is another cardinal 
principle of progr a ming ? namely, that the stu— 
dent is the final authority in determining 
whether or not the program is good. In 
traditional curriculum materials an "expert" 
often determines the final presentation, hut in 
programming, the approach is more student-centered. 
Programs are also more carefully aimed at a parti- 
cular ability- level of student, based on experi- 
mentation, not on opinion. 

7. The student is usually free to vary his own rate 
of learning. A student may work through a pro- 
gram rapidly or slowly. He is completely in- 
dependent of others in the class. Traditional 
methods such as lectures or motion pictures force 
every student to proceed at the same rate, which 
might be too fast for some and too slow for 



others 
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Programs are usually divided into two main types, 
depending on the kind of response demanded cf the student. The 
constructed- response type of program requires the student to 
write an answer to a question put before him by the programmer. 

(See Figure 1) The multiple-choice type of program requires 
the student to select one of a number of alternate answers to a 
given question. (See Figure 2) The constructed-response pro- 
gram asks the student to frame his own answer to an " open-ended 
question: the multiple-choice programs ask for a choice among 
alternate answers. The former clearly depends more upon the 
student's ability to recall data; the latter, on the ability to 
recognize it. 

There are two major techniques for programming sequences 
that are currently widely used. In one case, the material is arranged 
in a single-ordered sequence and every student must proceed from 
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the first through the last item. This is known as linear programming. 
In other cases, more than one sequence or route through the mater- 
ial is arranged, the the student follows the sequence determined by 
hist own answers. For example, a correct response to one question 
may lead down a route that skips several questions, while an in- 
correct reply produces a route on which each of these questions 
must be answered. This practice of providing alternate routes 

through the program is called branching . 

COMPARATIVE RESEARCH 

If one were to search the literature to answer our question, 

"How effective is programmed instruction in the teaching of reading?" 
he would be hard put to find much serious research bearing directly 
on this question. In fact, since this question remains pretty much 
unanswered, I think that the best thing to do is to go back to my 
presentation of last year in which I stated that the biggest and 
best controlled study on teaching of reading by programmed instruc- 
tion was done by Ruddell which was one of the U.S.O.E. sponsored 
first-grade studies (7^) (14). Ruddell was really interested in 
seeing if certain linguistic-type supplements to both basal reading 
texts and programmed reading books would aid in reading instruction, 
but he also included in his study one group of classrooms which used 
the Sheldon Basic Readers and another set of classrooms which used 
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the programmed reading series bySulxivan and Associates. (See 
Figure 1) By looking at Table 1 we can see that there really isn't 
much difference between reading achievement scores at the end of 
first grade in the two groups. 

A small study was done by Banna tyne at the Word Blind Study for 
Dyslexic Children in London using linear programs to teach punctua- 
tion and time telling (JL). These children were matched on age, 
sex and non-verbal ability and reading age (the lowest reading age 
was 7.0). Bannatyne found that both the teaching machine group and 
the orthodox teaching group gained significant knowledge but that 
"it can be concluded that within the limits of this experiment, 
teaching machines teach second year junior children no better or 
worse than teachers in an orthodox teaching situation." 

In a master thesis study done by Sifgler in 1967 an attempt was 
made to measure growth in reading of 36 mildly remedial high school 
readers. After nine 45 minute sessions using the Lessons for Self 
Instruction Basic Skills (See Figure 2), students had lost a tenth 
of a year in scores on the Gates Reading Survey between Form 1 and 
Form 2. I cite this study not to show that you gc backwards using 
programmed instruction but rather some of the methodological flaws 
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and difficulties in trying to measure gains on programmed instruc- 
tion, In this study we had a short training time coupled with an 
insensitive instrument. The Gates Reading Survey is a good gross 
screening device, but with a range of third grade through tenth 
grade in 21 items of the comprehension test, you don't get much 
reliability. 

When studies had been done in other subject matter areas in 
which the exact same content had been programmed and presented in 
text form, we often see little difference (5) (13). My last year's 
paper before the IRA reported some summaries of other studies with 
positive and no-difference results (X)* It is difficult to find 
studies with negative results as the investigators tend not to 
write up failures (assumming they are interested in the "experimental 
method"), but we have one near admission in a project done in the 
New York City Schools. 



"Gotkin and others became involved with this popula- 
tion in 1963 in the Reading Improvement Project of the 
Center for Programmed Instruction. During the two years 
with this project, they wrote and tested programmed in- 
struction lessons directed at teaching a number of skills 
designed to upgrade the reading ability and subject-matter 
vocabulary of seventh and eighth graders who were reading 
at the fourth grade level. In terms of the goals of the 
project they failed to produce a significant amount of pro- 
grammed materials capable of modifying the critical aspects 
of the reading behavior of their target population. " ( 9 ) 
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Even though the effectiveness of programmed instruction 
over conventional methods is yet to he demonstrated, there has 
230011 some interesting research on parts of the programmed learning 
process. Gillooly in 1968 has shown after a review of his own and 
other studies, that if you expect to have the student learn to make 
constructed responses (the criterion task of writing the answer) 
then you must train him with a cons tructed-response- type program, 
but if the student is only expected to make a selection of multiple 
choices, then training on a multiple choice program is satisfactory. 
(13) This superiority of constructed responses is particularly im- 
portant in teaching of novel terms. Reading teachers might make 
use of this information by providing ample experiences of writing 
words in vocabulary lessons. 



A HISTORICAL VIEW 

What then keeps programmed instruction alive? Part of 
the answer might be found in ancient dreams. Dale in his article 
"Historical Setting of Programs" in the 1967 NSSE Yearbook cites 
an interesting passage from Edward Thorndike written in 1912 giving 
us almost a prescription for programmed books s (J3) 
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"Books could be written giving data, directions 
for experiments and problems with the data, and 
questions about the inferences. The students could 
be instructed to read each helping piece of infor- 
mation- suggest questions and the like only after 
he had spent a certain amount of time in trying to 
do for himself what he was directed to do. ..if by 
a miracle of mechanical ingenuity a book could be so 
arranged that only to him that had done what was 
directed on page 1, would page 2 become visible and 
so on. Much that now requires personal instruction 
could be managed by print. " 



Thorndike then went on to give a classic argument for 
automation: "A human being should not be wasted in doing what 

40 sheets of paper or 2 phonographs could do." Dale then discussed 



the similarity between some modern programmed instruction books 

and the catechism- type of question and answ*. books used by the 
ancient Greeks. 



For anyone who has studied a history of education, it 
becomes harder and harder to see "new” ideas. The stress on clear 
educational objectives did not begin with Skinner or even Mager. 

A decade earlier. Bloom and ICrathwohl were carrying the banner and 
before them were Tyler and Charters. This notion of clarifying 
educational objectives also stretches back to the Greeks and 
orobably before them if we had any literature on pedagogy. 
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Programmed instruction seems to have burst upon us in 
relative recent times, but it has had an uncertain and unsteady 
progress* Since Pressey first published his article in 1926 very 
few other works were done in the area of teaching machines or pro- 
grammed instruction, with the exception of a few of his students. 
The present cycle of activity began in 1954 with Skinner* s article 
and interest in teaching machines and programmed learning seems to 
have reached some kind of peak about 1963* 

Corey in his 1967 NSSE article has given us a good index 
of activity by simply counting the number of entries in the 
Educational Index for two-year periods beginning in 1959. (2) 

He carried his count through 1965 and I extended two more years. 

We find that by combining the two subject matter headings of 
Programmed Teaching and Teaching Machines, that in 1959—61 there 
were 130 articles. In a two-year period of 1961-63 there were 440 
articles. This has steadily declined and in the 1963-65 period 
there were 342 articles but during 1965-6? the number had dropped 
to 232. (See Table 2) I even found a drop between the one-year 
period of 196S-66 and 1966-67. Coxey found a similar curve in 
entries in the Psychological Abstracts with the peak year being 
1964. I found a similar though later curve in the Subject Guide 
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to Books in Print, U.S. A . issues of 1966 and 1967 (See Figure 3)* 

The list of Reading Programmed Learning materials in Textbooks in 
Print shewed no change between the years 1967 and 1968; there were 
8 series or individual programs (Se<; Figure 4). A larger number 
of programs, many from small publishing houses, can be found in 
Programmed Instruction Guide , compiled by Northeastern University. 
(18) 

Yet with all of these articles, the lack of evidence 
about the effectiveness of programmed instruction, particularly in 
the field of reading, is striking. Silverman was able to write a 
whole chapter entitled, "Reading and Related Verbal Learning" in 
the NEA sponsored book Teaching Machines and Programmed Learning II 
without getting down to the comparison or effectiveness problem. 

(16) The National Society for the Study of Education put out an 
entire yearbook called Programmed Instruction in 1967 which has 
very little about the effectiveness of programmed instruction 
and nothing about the effectiveness of reading instruction. The 
NSSE Yearbook in the following year, 1968, put out a volume on 
reading entitled Innovation and Change in Reading Instruction in 
which programmed instruction was mentioned by the authors; none 
of them gave any data as to its effectiveness. (12 ) 

Publishers have some glowing testimonials and even 
quotes of small unpublished studies, but in the regular literature. 
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there is very little. Perhaps 
g rammed instruction when we do 



it is unfair to ask 
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supplementary instruction^ materials. But programmed instruction 
was born in the psychological laboratories where testing is the 
order of the day and somehow most of us had greater expectations 
for its empirical justification. 



PROGRAMMED INSTRUCTION USE IN INDUSTRY GREATER THAN SCHOOLS 



There is some evidence and a widespread feeling that 
programmed instruction is used more widely in industry than in the 
public schools. As evidence of this, one need only look at the 
membership of The National Society of Programmed Instruction which 
is the major professional organization in the field, and the type 
of articles published in their journal. (11 ) On the other hand, 
the Center for Programmed Instruction, which was largely education- 
oriented, merged into Columbia Teachers College and has more or 
less disappeared. A similar loss of educational interest is re- 
flected in the program and journal articles of the Division of 
Audio-Visual Instruction of NEA. 

I have wondered about the greater percentage of use of 
programmed instruction and the greater continuation of use in 
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industry than in the schools and would like to offer the follow- 



1. Training objectives are more limited and more specific. 

An airline is interested in having certain employees know 
how to read the weather code symbols, but in teaching about 
weather, a school doesn't know if it should stop at meteorology, 
physics, chemistry, industrial pollution or ecological effects 
once it starts talking about weather. Likewise the school 
feels that it has done a great job if a student starts reading 
about weather and gets a student to go on and learn about the 
influence of the moon on tides; but an industry has done a 
good training job if the employee learns the weather code 
quickly and gets back to work earning money for the company. 

2. Industry has no vested interested in the status quo . 



If a school superintendent decided to temporarily place half 
his teachers into curriculum development and writing programs, 
this would mean that the other half would be teaching twice 
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as many children. This situation would undoubtedly mean 
immediate dismissal of the superintendent? yet a training 
director in industry could make this kind of decision on 
Wednesday and have it implemented by the following Monday* 
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3. industry typically has a much less st able s tudent populatio n. 
The public school principal knows within a couple of per- 
centage points how many students will be at what stage of 
development a year or two in advance. In industry , model 
changes, market fluctuation, and technological advances all 
contribute to short term fluctuations. The training director 
may have 500 students attending classes one week and 1,000 
the next. He may have some classes at 10s 00 a.m. and some 

at 10s 00 p.m. He may hold some classes for 25 employees at 
the home office and some for three students in Tulsa. In 
short, the flexibility to package up the training and ship it 
anywhere at any time for any amount of students is worth a 
lot more to industry than to the schools. 

4. Trained teaching personnel is much more avai lable in the 
schools than in industry . Even with long-term teacher 
shortages in the public schools, there is a much more readily 
available supply of trained personnel for public school teach- 
ing than any industry can muster. In industry, training is 
often done by supervisors who were trained primarily in work 
experience and who often view teaching as an unpleasant occa- 
sional necessity. 
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I don*t wish to belabor the point of differences 
vaKroan 4nAie<-r*iai t^^inincr and public schools , but it does 
tend to answer the question of "Why is more programmed instruc- 
tion done in industry than in the schools?" It also points out 
the important "system" concept that any training method must be 
evaluated in light of the total training situation, and to simply 
ask "Does A teach better than B? M must be modified by a whole 
string of qualifications like who? when? and where? 

THE EVALUATION PROBLEM 

The whole problem of evaluation of curriculum materials 
or teaching methods is extremely important. Probably the best 
model of critical evaluation comes from the testing field where 
every published test is critically reviewed by experts in the 
field in Buros* Mental Measurements Yearbooks . But even this, 
as valuable as it is, falls short of empirical validation or 
comparison. In other words, the experts look at the tests and 
the manuals and make judgments based on experience or knowledge 
of testing. It would be more valuable if they or several in- 
dependent sources did an item analysis, comparative validations 
and other statistical verification based on actual administration 
of the test to specific populations. 
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The reading field recently saw a major effort in com 



parative evaluation in the USOE sponsored First Grade Studies in 
which 27 independent investigators tried out two or more methods 
of teach i ng beginning reading on moderately large populations 



using the same reading tests as achievement measures. The Ruddell 
study mentioned earlier was the only one of these to use pro- 
grammed instruction. I fervently hope that when money for educa- 
tional research is again available, that not only will the First 
Grade studies be replicated, refined, and extended, but we will 
investigate upper levels of reading instruction in the same 
controlled objective manner. 



Many members of the American Educational Research Asso- 
ciation have long been concerned with comparative evaluation of 
teaching methods and materials. At their recent conference , 
February 1968, a major symposium was devoted to this topic and 
Louise Tyler presented a set of "Recommendations for Curriculum 
and Instruction Materials" which grew out of an ongoing project at 
the University of California at Los Angeles. These recommendations 
had much to say that bears on objective evaluation, not the least 
of which is the calling on publishers to produce a manual similar 
to a test manual which specifies not only objectives, but such 



evaluation procedures ass (17) 
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E2 Manuals should clearly distinguish between 
v j />»r5 ni^oc An te^ 3bn\it ef f ectiveness s 

(a) internal evidence 

(b) external evidence 

Internal refers to features that can be revealed 
through visual inspection of study of materials. 
External refers to tryouts, revisions, etc. 

# # # 

E4 Effectiveness of programs should be reported in 
terms of program objectives as well as unintended 
outcomes • 



# # # 

E5 Curriculum and instruction materials should be 
evaluated in relation to different types of students, 
e.g., intellectual level, sex, age, socio-economic. 



These lofty requests call to mind the efforts of the 
AERA-APA-nAVI Joint Committee on Programmed Instruction and Teach- 
ing Machines who issued the Recommendations for Reporting the 
Effectiveness of Programmed Instruction Materials^ ( 10) These 

recommendations, like Tyler's, are essentially what the educational 
researchers and the better informed curriculum purchasers would 
like to have. However, no publisher has even come close to 
supplying the requests for the type of manual called for by either 
group. Perhaps someday professional organizations such as the IRA 
can arouse their membership into demanding that publishers pro- 
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duce such manuals to accompany instructional materials or else 
they will refuse to purchase. It is a little unrealistic to 
expect publishers to start publishing these expensive manuals if 
they don’t have to or, I might add, if nobody is willing to pay 
for them. 



CONCLUSION 

In returning to our original topic, "How effective is 
Programmed Instruction in Teaching Reading?", I am reminded of an 
incident several years ago in which I was discussing a report on 
educational television with a prominent educator. The educator 
said after looking at numerous tables and graphs, "Well, TV’s no 
more effective than an ordinary classroom teacher. " He could have 
even justified the statement by saying there's no statistically 
significant difference between children taught by a teacher and 
those taught by instructional television. The significance, how- 
ever, was not of the .01 or the .05 nature, rather it had an en- 
tirely different sort of significance in which, with educational 
television, we had one teacher teaching a hundred thousand children 
as opposed to the ordinary classroom teacher who taught 30. 



I am not proposing that programmed instruction is here 
to replace teachers r but we do have numerous studies which show 
that it does teach some things as well as a teacher , which means 
that the teacher can then do something else. For example, we 
can have no teacher or paraprofessionals doing part of the in- 
struction while the teacher diagnoses a weakness or motivates an 
underachiever • 

Somehow or other I can*t believe that a method which 
incorporates such saintly goals as specifying objectives, allow- 
ing for individual differences in learning rate, allowing for 
great diversity of subject matter being taught (even if the 
teacher doesn*t know the subject matter) and provides for student 
interest through a variety of instructional techniques, is going 
to be allowed to fade away completely. In fact, there* s evidence 
that programmed instruction is being incorporated into the new 
supernova of Computer-Assisted Instruction and the minor novas of 
individually Prescribed Instruction and the Systems Approach. 

I expect that in a few years someone will be able to 
do an Education Index count of articles and show a rise and 
decline for CAI. But in the meantime, one of the units that is 
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rapidly becoming a staple in the teacher’s bag of tricks is 
programmed instruction. 1 personally have learned a lot about 
teaching from studying teaching machines and programmed instruc- 
tion. It has greatly influenced my thinking and outlook and I 
hope in some small way all of this plethora of article-writing 
activity has somehow helped that classroom teacher to do a better 
job with her children. 
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Figure 1. Sample page from Programmed Reading Book 1 , A 
Sullivan Associates Program. McGraw-Hill Book Company, 1963. 




Figure 2. Sample page from Following Directions. A Programmed 
Reading Study by Miles Midloch in the Lessons for Self Instruction 
Series, California Test Bureau, Monterey, California, 1963. 
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Mary Smith, studying for a test, had no time to go out 
for dinner. She took a packaged, precooked frozen chicken 
dinner from the refrigerator. 

On the box she found these directions: 

(1) Preheat oven to 400° (hot oven). * 

(2) Remove dinner from box. 

(3) Do not remove or tear aluminum foil which 
chould stay sealed to keep the steam and 
juices from drying out. 

(4) Place dinner in preheated oven for 15 minutes, 
then remove it quickly. 

(5) Fold back foil to expose chicken but not veg- 
etable and potatoes. 

(6) Return dinner to oven for another 10 minutes 
to complete cooking and brown the meat. 

When Mary sat down to eat her dinner, she found that the 
chicken was pale outside and underdone inside. Did Mary 
omit any step? Did she follow directions in a wrong order? 
(Her oven was in good order. Whatever happened was Mary’s 
fault.) 

Mary omitted Step 2 of the directions. (No. 65) 

She removed the foil before first 

putting the dinner into the oven. (No. 69) 

Mary omitted Step 6 of the directions. (No. 73) 



62 $ Am put /t&W'S? 

q No. If £ is over o. in your code, something is 

[€.& J wrong. Did you follow the directions given in 
o J No. 67 carefully? You did not! An important 

part of it escaped you. 

Return to No. 67 and read the directions again. Compare 
your code with each step of the directions. 
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* These are books about programmed instruction. 
** 



No books listed prior to 1951. 



Figure 4. List of Heading Programmed Learning Materials in 

Textbooks in Print 1968* 



1. Allen, R. V. & Claryce. Language experiences in reading, 
levels 1-3, gr. k-3, 1967, EBE Corp. 
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basic readers, 1963, Ginn. 



3. Bostwick, Gracecarol & Miles Midloch. Lessons for Self 
Instruction in Basic Skills, 1964-1966, Cal Test/McGraw-Hill, 

4. Carroll, Luch. Programmed phonics, 1966, ,Educ. Pub. 



5. Educational Development Corp. Help yourself to read, write, 
and spell, 1966, Ginn. 



6. Glassman, J. Programmed reading, 1966, Globe. 

7. Loretan, Joseph O. and New York City Schools staff. Building 
reading power, 1964, Merrill. 

8. Sullivan Associates. Programmed reading ser. gr. 1-3, 
1963-1965, Webster/McGraw-Hill. 



*Category 141 Readers and Category 142 Reading Skills Under Pro^ 

grammed Learning. Materials show no increase or decrease from 
1967. These are programmed materials for students. 



TABLE 1 



PROGRAMMED READING COMPARED WITH BASAL READERS AT 

THE END OF 1st GRADE 

(Data taken from Ruddell 1965) 



Group Reading Test Means 



Buchanan Sheldon 

Programmed Reading Basic Readers 



Stanford Achievement Test 
Primary I (N=about 132 per cell) 
Grade Scores 



Paragraph Meaning 


1.6 


1.7 


Word Reading 


1.8 


1.7* 


Word Study Skills 


1.7 


1.7 


Spelling 


1.7 


1.7 



Individual Reading Test Means 
Raw Scores (N=about 44 per cell) 



Gilmore Oral Accuracy 


16.6 


17.7 


Gilmore Oral Rate 


46.8 


51.8 


Gates Word - Oral (Words 






not selected for phonic 






regularity) 


11.8 


1C.S 


Phonetically Regular Words 






Oral Reading Test 


9.1 


5.4* 



*Ruddell found raw scores with statistically significant 
differences between means at .05 level. 



Table 2 



Number of Entries in Education Index . 1959-67* 



Year 


Programmed 

Teaching 


Teaching 

Machines 


TOTAL 


July, 1959- 
June, 1961 


51 


79 


130 


July, 1961- 
June, 1963 


321 


119 


440 


July, 1963- 
June, 1965 


313 


29 


342 


July, 1965- 
June, 1967 


186 


46 


232 



*Adapted from Corey (2) and extended by Fry 



